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1.0  INTRODUCTION 


1.1  ORGANIZATION  OF  THIS  DOCUMENT 

This  report  details  the  partieipation  of  the  Naval  Researeh  Laboratory  (NRL)  Man-Portable 
Eleetromagnetie  Induetion  Array  for  UXO  Deteetion  and  Diserimination,  or  TEMTADS  Man- 
Portable  (MP)  2x2  Cart,  in  the  Environmental  Seeurity  Teehnology  Certifieation  Program 
(ESTCP)  Munitions  Response  Eive  Site  Demonstration  at  the  Central  Impaet  Area  (CIA), 
Massaehusetts  Military  Reservation  (MMR),  loeated  at  Camp  Edwards,  MA  in  2013.  The  2013 
demonstration  eompleted  the  effort  begun  in  2012.  To  limit  the  repetition  of  information,  Study- 
and  site-  speeifie  information  that  is  presented  in  the  ESTCP  Eive  Site  Demonstrations  Plan  [1] 
is  noted  and  not  repeated  in  this  doeument. 

1.2  STUDY  BACKGROUND  AND  OBJECTIVES 

Please  refer  to  the  ESTCP  Eive  Site  Demonstrations  Plan  for  the  MMR  CIA  [I]. 

1.3  SPECIFIC  OBJECTIVES  OF  DEMONSTRATION 

As  part  of  the  ESTCP  Eive  Site  Demonstrations,  NRL  and  NAVEA  Geophysies  eondueted  a 
eued  elassifieation  survey  on  the  remaining  1.75  aeres  of  the  3-aere  man-portable  subarea 
seleeted  from  within  the  330-aere  CIA  demonstration  site.  The  2012  NRL  survey  investigated 
1,005  anomalies  in  the  northern  1.25  aeres  of  the  man-portable  area.  Cued  data  eolleetion  was 
eondueted  for  the  remaining  1,429  anomalies  previously  identified  from  an  EM61-MK2  eart 
survey  eondueted  in  2012  by  National  Guard  Bureau  eontraetor  TetraTeeh.  This  survey  used  the 
NRL  TEMTADS  MP  2x2  Cart  (MP  System)  operating  in  litter-earry  mode.  Charaeterization  of 
the  system  responses  to  the  Targets  of  Interest  (TOIs)  was  based  on  previously  aequired 
TEMTADS  referenee  data.  These  data  were  eolleeted  in  aeeordanee  with  the  overall  study 
objeetives  and  demonstration  plan.  Unlike  the  2012  demonstration,  elassifieation  analysis  of  the 
interrogated  anomalies  was  eondueted  as  well.  This  doeument  deseribes  the  results  of  the 
demonstration. 

2.0  TECHNOLOGY 

2.1  TECHNOLOGY  DESCRIPTION 

2.1.1  TEMTADS/3D  EMI  Sensors 

The  original  design  of  the  MP  system  utilized  the  standard  TEMTADS  Eleetromagnetie 
Induetion  (EMI)  sensor.  Based  on  the  results  of  the  MP  system  demonstration  at  the  Aberdeen 
Proving  Ground  (APG)  Standardized  UXO  Test  Site  in  August,  2010  [2,3],  revision  of  the 
sensor  teehnology  was  indieated.  A  modified  version  of  the  sensor  element  was  designed  and 
built,  replaeing  the  single,  vertieal-axis  reeeiver  eoil  of  the  original  sensor  with  a  three-axis 
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receiver  cube.  These  receiver  cubes  are  similar  in  design  to  those  used  in  the  second-generation 
Advanced  Ordnance  Locator  (AOL)  and  the  Geometries  MetalMapper  (ESTCP  MR-200603) 
system  with  dimensions  of  8  cm  rather  than  10  cm.  The  CRREL  MPV2  system  (ESTCP  MR- 
201005)  uses  an  array  of  five  identical  8  cm  receiver  cubes  and  a  circular  transmitter  coil.  The 
new  sensor  elements  are  designed  to  have  the  same  form  factor  as  the  original,  aiding  in  system 
integration.  A  TEMTADS/3D  coil  under  construction  is  shown  in  Figure  2-1. 


Figure  2-1  -  Individual  TEMTADS/3D  EMI  sensor 
with  3 -axis  receiver  under  constmction. 

Minor  modifications  were  made  to  the  AOE  control  and  data  acquisition  infrastructure  to  make  it 
compatible  with  our  deployment  schemes.  Decay  data  are  collected  with  a  500  kHz  sample  rate 
until  25  ms  after  turn  off  of  the  excitation  pulse.  These  data  result  in  a  decay  of  12,500  points; 
too  many  to  be  used  practically.  These  raw  decay  measurements  are  grouped  into  122 
logarithmically-spaced  “gates”  with  center  times  ranging  from  25  |us  to  24.35  ms  with  5%  widths 
and  are  saved  to  disk. 

2,1,2  Application  of  the  Technology 

Application  of  this  technology  was  straightforward.  A  list  of  initial  target  positions  was 
developed  from  EM61-MK2  data  collected  by  TetraTech  in  2012.  A  plastic  pin  flag  was 
manually  placed  over  each  anomaly  location  prior  to  cued  data  collection.  The  MP  System  was 
then  positioned  over  each  target  flag  in  turn.  At  each  position,  the  transmitter  for  each  array 
sensor  was  fired  in  sequence  and  decay  data  were  collected  from  all  twelve  receiver  coils  for 
each  transmitter  excitation.  These  data  were  then  stored  electronically  on  the  data  acquisition 
computer.  Prior  to  moving  to  the  next  target,  the  operator  evaluated  the  data  by  running  a  single¬ 
dipole  fitting  routine  to  extract  target  parameters  (dipole  fit  location  and  magnetic  principal 
polarizability  decays).  Additionally,  the  operator  reviewed  a  display  of  the  4  monostatic,  3-axis 
signal  amplitude  decays  and  transmit  current.  If  the  fit  location  of  the  anomaly  was  not 
sufficiently  close  to  the  array  center  (horizontally)  or  other  data  quality  metrics  were  not  satisfied 
(e.g.  peak  signal  amplitude),  the  operator  had  the  option  to  collect  additional  data  for  the  target 
prior  to  leaving  the  target  location.  The  EMI  data  were  transferred  to  the  off-site  analyst  at  least 
once  a  day  for  processing  and  analysis. 
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2.1,3  Development  of  the  Technology 


The  MP  System  is  a  man-portable  four-element  transient  EMI  (TEM)  system  designed  and  built 
by  NRL  with  funding  from  ESTCP.  The  MR-200909  project  goal  was  to  transition  the  TEM 
sensor  technology  of  the  TEMTADS  towed  array  (ESTCP  Project  MR-200601)  to  a  more 
compact,  man-portable  configuration  for  use  in  more  limiting  terrain.  Preliminary  testing  of  the 
initial  system  configuration  [4]  found  that  for  high  SNR  (>  30)  targets  one  measurement  cycle 
provides  enough  information  to  support  classification.  Eor  deeper  and/or  weaker  targets,  more 
robust  estimates  of  target  parameters  were  obtained  by  combining  two  closely-spaced 
measurements.  Two  measurements  per  anomaly  were  typically  made  proactively  to  avoid  the 
potential  need  to  revisit  a  target  a  second  time.  As  part  of  project  MR-200909,  a  demonstration 
was  conducted  to  rigorously  investigate  the  capabilities  of  this  new  sensor  platform  for 
unexploded  ordnance  (UXO)  classification  in  a  cued  data  collection  mode  at  the  APG 
Standardized  UXO  Test  Site  in  August,  2010  [5].  Those  results  indicated  that  the  inversion 
performance  of  the  system  was  not  comparable  to  that  of  the  full  TEMTADS  array  for  lower 
SNR  targets  due  to  the  limits  of  the  smaller  data  set  (fewer  looks  at  the  target).  A  modified 
version  of  the  EMI  sensor  was  designed  and  built,  replacing  the  single,  vertical  axis  receiver  loop 
of  the  original  coil  with  a  tri-axial  receiver  cube.  These  receiver  cubes  are  identical  in  design  to 
those  used  in  the  CRREE  MPV2  system  (ESTCP  MR-201005).  The  new  sensor  elements  were 
designed  to  have  the  same  form  factor  as  the  originals,  aiding  in  system  fabrication.  The 
completed  MP  System  has  been  demonstrated  as  part  of  the  ESTCP  Munitions  Response  Eive 
Site  Demonstrations  at  several  sites  [6,7,8]. 

2.2  ADVANTAGES  AND  LIMITATIONS  OF  THE  TECHNOLOGY 

The  MP  System  was  originally  designed  to  offer  similar  cued-mode  production  rates  to  those 
seen  for  larger,  vehicular-towed  advanced  EMI  sensors  while  able  to  operate  in  difficult  terrain 
and  treed  areas  that  the  larger  systems  cannot  access.  With  the  upgraded  TEMTADS/3D 
sensors,  similar  performance  was  achieved  with  similar  classification-grade  data  quality.  The 
MP  array  is  80  cm  on  a  side  and  mounted  on  a  man-portable  cart.  Terrain  where  the  vegetation 
or  topography  interferes  with  passage  of  a  cart  of  that  size  will  not  be  amenable  to  the  use  of  the 
system.  Eor  this  demonstration,  a  litter-mode  configuration  was  successfully  demonstrated  to 
alleviate  brush-clearing  and  ground-clearance  issues  at  this  site. 

There  is  a  limiting  anomaly  density  above  which  the  response  of  individual  targets  cannot  be 
separated  individually.  We  have  chosen  relatively  small  sensors  for  this  array  which  help 
mitigate  this  problem  but  we  cannot  eliminate  it  completely.  Recent  developments,  including 
solvers  designed  for  classification  in  multiple-object  scenarios  such  as  SAIC’s  multi-target 
solver,  [9]  are  being  evaluated  and  their  performance  characteristics  in  cluttered  environments 
determined. 
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3.0  PERFORMANCE  OBJECTIVES 


Performance  objectives  for  the  demonstration  are  given  in  Table  3-1  to  provide  a  basis  for 
evaluating  the  performance  and  costs  of  the  demonstrated  technology.  Overall  project  objectives 
are  given  in  the  overall  demonstration  plan  generated  by  ESTCP.  Since  this  is  a  classification 
technology,  the  performance  objectives  focus  on  the  second  step  of  the  UXO  survey  problem;  we 
assume  that  the  anomalies  from  all  targets  of  interest  have  been  detected  and  included  on  the 
target  list  we  worked  from.  Since  the  development  of  the  Demonstration  Plan  Supplement  and 
the  Objectives  laid  out  within,  events  have  transpired  such  that  dig  results  may  not  become 
available  to  allow  for  scoring  the  classification  results  presented  in  this  report.  As  such,  the 
requisite  data  for  evaluating  the  objectives,  so  noted  individually,  may  not  become  available  at  a 
future  point. 

3.1  OBJECTIVE:  REPEAT ABILTY  OF  INSTRUMENT  VERIFICATION  STRIP 
MEASUREMENTS 

This  objective  verifies  that  the  sensor  system  is  in  good  working  order  and  is  collecting 
physically  valid  data  each  day.  The  items  emplaced  in  the  Instrument  Verification  Strip  (IVS) 
were  surveyed  twice  daily.  The  amplitude  of  the  derived  response  coefficients  and  fit  location 
for  each  emplaced  item  were  compared  to  the  running  average  of  the  demonstration  for 
reproducibility. 

3.1.1  Metric 

The  reproducibility  of  the  measured  response  of  the  sensor  system  to  the  emplaced  items  defines 
this  metric. 

3.1.2  Data  Requirements 

The  tabulated  fit  parameters  for  the  data  corresponding  to  each  emplaced  item  in  terms  of 
derived  response  coefficients  and  fit  location. 

3.1.3  Success  Criteria 

The  objective  was  considered  met  if  the  root-mean-squared  (RMS)  amplitude  variation  of  the 
derived  response  coefficients  and  fit  location  were  less  than  10%. 

3.2  OBJECTIVE:  CUED  INTERROGATION  OF  ANOMALIES 

To  collect  EMI  data  of  the  highest  quality  for  UXO/clutter  classification,  the  anomaly  must  be 
illuminated  along  its  three  principal  axes.  To  ensure  this,  the  data  collection  pattern  (in  this  case 
the  TEMTADS  array)  must  be  positioned  directly  over  the  center  of  the  anomaly. 
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Table  3-1  -  Performance  Objectives  for  this  Demonstration 


Performance 

Objective 

Metric 

Data  Required 

Success  Criteria 

Data  Collection  Objectives 

Instrument 

Verification  Strip 
(rVS)  Results 

System  responds 
consistently  to 
emplaced  items 

Daily  IVS  data 

<  10%  RMS  variation 
in  P  amplitudes  and  in 
fit  location 

Cued  interrogation  of 
anomalies 

Instrument  position 

Cued  Data 

100%  of  anomalies 
where  the  center  of 
the  instrument  is 
positioned  within  40 
cm  of  actual  target 
location 

Analysis  and  Classification  Objectives 

Maximize  correct 
classification  of  TOI 

Number  of  TOI 
retained 

Ranked  anomaly  lists 

Results  of  intrusive 
investigation 

Approach  correctly 
classifies  all  TOI 

Maximize  correct 
classification  of 
non-TOI 

Number  of  false 
alarms  eliminated 

Ranked  anomaly  lists 

Results  of  intrusive 
investigation 

Reduction  of  clutter 
digs  required  by 
>75%  while  retaining 
all  TOI 

Specification  of 
no-dig  threshold 

Probability  of  correct 
classification  of  TOI 
and  number  of  false 
alarms  at  operating 
point 

Specified  threshold 

Results  of  intrusive 
investigation 

Threshold  specified  to 
achieve  criteria  above 

Minimize  number  of 
anomalies  that  cannot 
be  analyzed 

Number  of  anomalies 
that  must  be  classified 
as  “Unable  to 

Analyze” 

Extracted  target 
parameters 

Reliable  target 
parameters  can  be 
estimated  for  >95% 
of  anomalies. 

Correct  estimation  of 
target  parameters 

Accuracy  of  estimated 
target  parameters  for 
seed  items 

Extracted  target 
parameters 

Results  of  intrusive 
investigation 

Polarizabilities  ±  20% 

X,  Y  <  15  cm  (la) 

Z  <  10  cm  (la) 
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3.2,1  Metric 


The  metric  for  this  objective  is  the  percentage  of  anomalies  that  were  within  the  acceptable 
distance  of  the  center  of  the  instrument  during  data  collection  from  the  actual  target  location. 

3.2.2  Data  Requirements 

After  preliminary  data  collection  and  any  reacquisition  cycles  were  complete,  the  offset  from 
array  center  to  inverted  target  location  were  determined  for  each  anomaly.  The  offset  distance 
was  required  to  be  less  than  40  cm. 

3.2.3  Success  Criteria 

The  objective  was  considered  met  if  the  center  of  the  instrument  was  positioned  within  40  cm  of 
the  anomaly  fit  location  for  100%  of  the  cued  anomalies.  Exceptions  were  made  for  anomalies 
which  were  partially  or  completely  occluded  by  obstructions  such  as  trees  or  were  located  in 
areas  where  intrusive  investigation  were  not  planned  (e.g.  roads).  Several  earthen  berms  formed 
from  bulldozed  dirt,  vegetation,  and  MEC  were  also  excluded  from  the  demonstration  for  not 
being  good  candidates  for  classification.  Anomalies  located  within  a  known  test  plot  were  also 
excluded  at  the  request  of  site  personnel. 

3.3  OBJECTIVE:  MAXIMIZE  CORRECT  CLASSIFICATION  OF  TOI 

This  is  one  of  the  two  primary  measures  of  the  effectiveness  of  the  classification  approach.  By 
collecting  high-quality  data  and  analyzing  those  data  with  advanced  parameter  estimation  and 
classification  algorithms  we  should  be  able  to  classify  the  targets  with  high  efficiency.  This 
objective  concerns  the  component  of  the  classification  problem  that  involves  correct 
classification  of  TOI.  Since  the  development  of  this  objective  as  part  of  the  Demonstration  Plan 
Supplement,  events  have  transpired  such  that  dig  results  may  not  become  available  to  allow  for 
scoring  the  classification  results  presented  in  this  report.  As  such,  the  requisite  data  for 
evaluating  this  objective  may  not  become  available  at  a  future  point. 

3.3.1  Metric 

The  metric  for  this  objective  is  the  number  of  items  on  the  MP  System  anomaly  list  that  were 
correctly  classified  as  TOI. 

3.3.2  Data  Requirements 

A  ranked  anomaly  list  was  prepared  for  the  targets  on  the  MP  System  anomaly  list. 

3.3.3  Success  Criteria 

For  the  purposes  of  this  demonstration,  the  objective  was  considered  met  if  all  of  the  TOI  were 
correctly  labeled  as  TOI  on  the  ranked  anomaly  list.  This  is  a  more  stringent  criterion  than 
required  for  the  site  clean-up  objectives  listed  in  the  Source  Removal  Work  Plan.  It  was  possible 
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to  achieve  results  adequate  for  the  site-speeifie  eleanup  objeetives  but  below  the  100%  eorreet 
elassifieation  specified. 

3.4  OBJECTIVE:  MAXIMIZE  CORRECT  CLASSIFICATION  OF  NON-TOI 

This  is  the  seeond  of  the  two  primary  measures  of  the  effeetiveness  of  the  elassifieation 
approaeh.  By  eolleeting  high-quality  data  and  analyzing  those  data  with  advaneed  parameter 
estimation  and  elassifieation  algorithms  we  expeet  to  be  able  to  classify  the  targets  with  high 
effieiency.  This  objective  eoncerns  the  eomponent  of  the  elassifieation  problem  that  involves 
false  alarm  reduetion.  Sinee  the  development  of  this  objeetive  as  part  of  the  Demonstration  Plan 
Supplement,  events  have  transpired  sueh  that  dig  results  may  not  beeome  available  to  allow  for 
seoring  the  elassifieation  results  presented  in  this  report.  As  sueh,  the  requisite  data  for 
evaluating  this  objeetive  may  not  beeome  available  at  a  future  point. 

3.4.1  Metric 

The  metrie  for  this  objeetive  is  the  number  of  items  on  the  MP  System  anomaly  list  that  were 
correetly  elassified  as  non-TOI. 

3.4.2  Data  Requirements 

A  ranked  anomaly  list  was  prepared  for  the  targets  on  the  MP  System  anomaly  list. 

3.4.3  Success  Criteria 

The  objeetive  was  eonsidered  met  if  more  than  75%  of  the  non-TOI  items  were  correetly  labeled 
as  non-TOI  while  retaining  all  of  the  TOI  on  the  dig  list. 

3.5  OBJECTIVE:  SPECIFICATION  OF  NO-DIG  THRESHOLD 

In  a  retrospeetive  analysis  as  was  performed  in  this  demonstration,  it  is  possible  to  tell  the  true 
classification  capabilities  of  a  elassifieation  proeedure  based  solely  on  the  ranked  anomaly  list 
submitted.  In  a  real-world  seenario,  all  targets  may  not  be  dug  so  the  suceess  of  the  approaeh  will 
depend  on  the  ability  of  an  analyst  to  aeeurately  speeify  their  dig/no-dig  threshold.  Sinee  the 
development  of  this  objective  as  part  of  the  Demonstration  Plan  Supplement,  events  have 
transpired  sueh  that  dig  results  may  not  beeome  available  to  allow  for  scoring  the  elassifieation 
results  presented  in  this  report.  As  sueh,  the  requisite  data  for  evaluating  this  objeetive  may  not 
beeome  available  at  a  future  point. 

3.5.1  Metric 

The  probability  of  eorreet  elassifieation  of  TOI,  Pciass,  and  number  of  false  alarms,  Nfa,  at  the 
demonstrator-speeified  threshold  were  the  metries  for  this  objective. 
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3.5.2  Data  Requirements 

A  ranked  anomaly  list  with  a  dig/no-dig  threshold  indieated  was  prepared  for  the  MP  System 
anomaly  list. 

3.5.3  Success  Criteria 

The  objeetive  was  considered  met  if  more  than  75%  of  the  non-TOI  items  were  correctly  labeled 
as  non-TOI  while  retaining  all  of  the  TOI  at  the  demonstrator-specified  threshold. 

3.6  OBJECTIVE:  MINIMIZE  NUMBER  OF  ANOMALIES  THAT  CANNOT  BE 
ANALYZED 

Evaluation  of  anomalies  for  which  reliable  parameters  could  not  be  estimated  cannot  continue  in 
the  classification  analysis  pipeline.  These  anomalies  must  be  placed  in  the  dig  category  and 
reduce  the  effectiveness  of  the  classification  process. 

3.6.1  Metric 

The  number  of  anomalies  for  which  reliable  parameters  could  not  be  estimated  was  the  metric 
for  this  objective. 

3.6.2  Data  Requirements 

A  list  of  all  extracted  target  parameters  was  provided  as  part  of  the  results  submission  along  with 
a  list  of  those  anomalies  for  which  parameters  could  not  be  reliably  estimated. 

3.6.3  Success  Criteria 

The  objective  was  considered  met  if  reliable  parameters  were  estimated  for  >  95%  of  the 
anomalies  on  the  anomaly  list. 

3.7  OBJECTIVE:  CORRECT  ESTIMATION  OF  TARGET  PARAMETERS 

This  objective  involves  the  accuracy  of  the  target  parameters  that  were  estimated  in  the  first 
phase  of  the  analysis.  Successful  classification  is  only  possible  if  the  input  features  are  internally 
consistent.  The  obvious  way  to  satisfy  this  condition  is  to  estimate  the  various  target  parameters 
accurately.  Since  the  development  of  this  objective  as  part  of  the  Demonstration  Plan 
Supplement,  events  have  transpired  such  that  dig  results  may  not  become  available  to  allow  for 
scoring  the  classification  results  presented  in  this  report.  As  such,  the  requisite  data  for 
evaluating  this  objective  may  not  become  available  at  a  future  point. 

3.7.1  Metric 

Accuracy  of  estimation  of  target  parameters  was  the  metric  for  this  objective. 
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3.7,2  Data  Requirements 


The  estimated  parameters  for  the  seed  items  and  the  ground  truth  for  the  seed  items  are  required 
data. 

3.7,3  Success  Criteria 

The  objeetive  was  eonsidered  met  if  the  estimated  polarizabilities  were  within  ±  20%,  the 
estimated  X,  Y  loeations  were  within  15  em  (la),  and  the  estimated  depths  were  within  10  em 
(la). 

4.0  SITE  DESCRIPTION 

Please  refer  to  the  ESTCP  Live  Site  Demonstrations  Plan  for  the  MMR  CIA  [1]. 

4,1  GPS  CONTROL  POINTS 

There  is  one  permanent,  fixed  GPS  station  operational  with  eoverage  of  the  CIA,  named  “Pine 
Hill  Reset.”  The  station  is  loeated  near  Range  Control  on  a  hilltop.  This  station  was  used  as  the 
GPS  base  station  for  the  2012  demonstration.  Parsons,  as  part  of  the  demonstration  seeding 
effort,  used  this  station  to  establish  two  additional  control  points  in  the  direct  vicinity  of  the 
MetalMapper  sub-area  and  the  2012  IVS  location.  The  fixed  station  uses  a  different  radio 
configuration  that  we  do  and  had  to  be  altered  in  2012  to  communicate  with  our  GPS  equipment. 
Rather  than  interfere  with  ongoing  operations  onsite,  we  used  the  “Parsons  Base”  auxiliary 
control  points,  whose  accuracy  was  verified  during  Parsons  seeding  and  MetalMapper 
demonstration  efforts,  for  the  2013  demonstration.  The  details  of  the  three  base  station  points 
are  listed  in  Table  4-1. 

Table  4-1  -  GPS  Control  Points  in  or  near  the  MMR  CIA 


ID 

Easting  (m) 

Northing  (m) 

Latitnde  (deg) 

Longitnde  (deg) 

HAE  (m) 

Pine  Hill  Reset 

369,733.893 

4,618,606.567 

41“  42’  30.92393” 

-70“  33’  56.81178” 

51.270 

Parsons  Base 

372,075.871 

4,618,769.323 

41“  42’  37.56772” 

-70“  32’  15.63205” 

50.028 

QC  Pipe  Parsons 

372,067.165 

4,618,744.806 

41“  42’  36.76799” 

-70“  32’  15.98970” 

49.039 

Easting  and  Northing  are  expressed  in  UTM  Zone  19N  (meters) 
All  horizontal  locations  are  in  the  NAD83  datum 


5.0  TEST  DESIGN 

5.1  CONCEPTUAL  EXPERIMENTAL  DESIGN 

The  demonstration  was  executed  in  two  stages.  The  first  stage  characterized  the  MP  System 
with  respect  to  the  site-specific  geology.  The  background  response  of  the  demonstration  site,  as 
measured  by  the  MP  System,  was  characterized  throughout  the  data  collection  process. 
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The  second  stage  of  the  demonstration  was  the  cued  survey  of  the  remaining  fraction  of  the  Man- 
Portable  subarea  of  the  demonstration  site  using  the  MP  System.  The  array  was  positioned 
roughly  over  the  center  of  each  target  flag  and  a  data  set  collected.  Each  data  set  was  inverted 
using  the  data  analysis  methodology  discussed  in  Section  6.0,  and  estimated  target  parameters 
determined.  The  results  and  the  archive  data  were  submitted  to  the  Program  Office.  Using  the 
fit  results,  a  prioritized  dig  list  was  prepared  and  submitted  to  the  Program  Office. 

The  schedule  of  field  testing  activities  is  provided  in  Figure  5-1  as  a  Gantt  chart. 


Activity  Name 

Jul  2013 

14 

21 

MMR  CIA  TEMTADS  MP  Demonstration 

MP  2x2  Cart  Data  Collection 

14 

21 

Figure  5-1  -  Schedule  of  Field  Testing  Activities 

5.2  SITE  PREPARATION 

Please  refer  to  the  ESTCP  Five  Site  Demonstrations  Plan  for  the  MMR  CIA  [1]. 

An  Instrument  Verification  Strip  (IVS)  was  emplaced  by  Parsons  for  the  2012  demonstration 
season  and  subsequently  removed.  TetraTech  emplaced  a  new  IVS  which  was  available  for  the 
2013  demonstration.  The  details  of  the  IVS  are  given  in  Table  5-1. 

Table  5-1  -  Details  of  the  TetraTech  2013  Central  Impact  Area  IVS 


ID 

Description 

Easting  (m) 

Northing  (m) 

Depth 

(m) 

Inclination 

Orientation 

T-01 

155mm  Projectile 

372,039.889 

4,618,975.979 

0.23 

Horizontal 

Along  Track 

T-02 

105mm  Projectile 

372,037.379 

4,618,977.423 

0.20 

Horizontal 

Along  Track 

T-03 

Medium  ISO  #1 

372,043.516 

4,618,977.582 

0.18 

Horizontal 

Along  Track 

T-04 

Medium  ISO  #2 

372,043.782 

4,618,975.526 

0.18 

Horizontal 

Along  Track 

“  Easting  and  Northing  are  expressed  in  UTM  Zone  19N  (meters). 

’’  Depth  is  reported  to  center  of  the  items  and  is  calculated  based  on  a  stated  burial  to  1 5  cm  for  the  top  of  each  item 


and  nominal  diameter  of  each  item. 

5.3  SYSTEMS  SPECIFICATION 

This  demonstration  was  conducted  using  the  NRL  TEMTADS  MP  2x2  Cart. 

5.3.1  TEMTADS  MP  2x2  Cart 

The  MP  System  is  a  man-portable  system  comprised  of  four  of  the  TEMTADS/3D  EMI  sensors 
discussed  in  Section  2.1.1  arranged  in  a  2x2  array  as  shown  in  Figure  5-2.  The  direction  of 
travel  is  to  the  left  in  Figure  5-2  (left)  and  up  in  Figure  5-2  (right).  The  MP  System,  shown  in 
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Figure  5-3  (left)  at  Fort  Rueker,  AL,  is  fabrieated  from  PVC  plastic  and  fiberglass.  The  center- 
to-center  distance  is  40  cm  yielding  an  80  cm  x  80  cm  array.  The  array  is  deployed  on  a  set  of 
wheels  resulting  in  a  sensor-to-ground  offset  of  approximately  20  cm.  The  transmitter 
electronics  and  the  data  acquisition  computer  are  mounted  in  the  operator  backpack,  as  shown  in 
Figure  5-5.  The  MP  System  can  be  operated  in  two  modes;  dynamic  or  survey  mode  and  cued 
mode.  A  Global  Positioning  System  (GPS)  receiver  and  an  inertial  measurement  unit  (IMU)  are 
mounted  above  the  TEM  array  as  shown  in  Figure  5-3  (right).  Data  collection  is  controlled  in 
dynamic  mode  using  G&G  Science’s  EM3D  application  suite,  the  same  software  that  used  for 
the  Geometries  MetalMapper  systems.  In  cued  mode,  the  locations  of  the  anomalies  must 
already  be  known  and  flagged  for  reacquisition.  Custom  software  written  by  NRE  provides  cued 
data  acquisition  functionality. 


Figure  5-2  -  TEMTADS/3D  EMI  sensor  array  with  weather  cover  removed  (left).  Sketch  of  the  EMI 
sensor  array  showing  the  position  of  the  four  sensors.  The  tri-axial,  revised  EMI  sensors  are  shown 
schematically  (right).  The  direction  of  travel  for  the  array  and  the  orientation  of  the  sensor  cubes  are 
indicated. 


In  cued  mode,  the  operators  position  the  cart  over  each  anomaly  location  in  turn  and  collect  a  set 
of  TEM  data.  Geolocation  and  cart  orientation  are  monitored  and  recorded.  Eunctionality  to 
record  field  notes  is  provided.  If  anomaly  flagging  is  unavailable  or  undesirable,  it  is  possible  to 
load  a  list  of  virtual  flag  locations  into  the  vendor-provided  survey  controller  for  the  GPS  unit 
and  use  the  provided  interface  for  anomaly-to-anomaly  navigation.  This  has  been  tested  using  a 
Trimble  R8  GNSS  version  3  GPS  receiver  and  a  TSC2  survey  controller. 


Eigure  5-3  -  The  NRE  TEMTADS  Man-Portable  2x2  Cart  (left)  and 
TEMTADS  MP  2x2  Cart  with  GPS  Antenna  Tripod  (right) 
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Due  to  the  terrain  and  the  state  of  brush  clearance  in  the  work  area,  a  litter-carry  configuration 
was  used  to  carry  the  system  from  flag  to  flag,  as  shown  in  Figure  5-4.  While  significantly  easier 
to  navigate  the  site  with  than  the  wheeled  version,  the  additional  weight  and  complexity  of  the 
litter-carry  configuration  necessitates  a  third  operator. 

5,3,2  Data  Acquisition  User  Interface 

The  data  acquisition  computer  is  mounted  on  a  backpack  worn  by  one  of  data  acquisition 
operators,  shown  in  Figure  5-5  (left).  The  second  operator  controls  the  data  collection  using  a 
tablet  computer  which  wirelessly  (IEEE  802.1  Ig)  communicates  with  the  data  acquisition 
computer.  The  second  operator  also  manages  field  notes  and  team  orienteering  functions.  In 
Eigure  5-3  (left)  and  Eigure  5-4  (right),  a  data  collection  team  is  shown  in  action. 


Eigure  5-4  -  The  NRE  TEMTADS  Man-Portable  2x2  Litter-Carry  Configuration  (left)  and  the 
Litter-Carry  Configuration  at  MMR  in  2013  (right) 

The  tablet  PC  user  interface  is  shown  in  Eigure  5-5  (right).  In  litter-carry  mode,  a  third  person 
lifts  and  carries  the  front  of  the  system. 


Figure  5-5  -  (left)  TEMTADS  2x2  Electronics  Backpack,  (right)  Screenshot  of  Tablet  Computer  Interface 
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5.4  CALIBRATION  ACTIVITIES 


5.4.1  TEMTADS  Sensor  Calibration 

For  the  TEMTADS  family  of  platforms,  a  significant  amount  of  data  has  been  collected  with  the 
systems  as  configured  at  our  Blossom  Point  facility,  both  on  test  stands,  on  our  test  field  [10]  and 
at  APG  [11],  and  as  part  of  the  ESTCP  Eive  Site  Demonstrations,  [6,8,12-14],  and  at  MMR  [7]. 
These  data  and  the  corresponding  fit  parameters  provide  us  with  a  set  of  reference  parameters 
including  those  of  clear  background  (i.e.  no  anomaly  present). 

Daily  calibration  efforts  consisted  of  collecting  background  (no  anomaly)  data  sets  periodically 
throughout  the  day  at  quiet  spots  to  determine  the  intra-site  variation  in  system  response  to  the 
local  geology  and  to  monitor  the  system  noise  floor.  The  items  emplaced  in  the  IVS  were 
measured  twice  daily  to  monitor  the  variation  in  the  system  response.  Variations  were  expected 
to  be  within  10%  of  the  reference  values.  These  two  types  of  measurements  constituted  the  daily 
calibration  activities. 

5.5  DATA  COLLECTION  PROCEDURES 

5.5.1  Scale  of  Demonstration 

NRL  and  NAEVA  Geophysics  conducted  a  cued  classification  survey  on  the  remaining  1.75 
acres  of  the  3 -acre  man-portable  subarea  selected  from  within  the  330-acre  CIA  demonstration 
site.  The  2012  NRL  survey  investigated  1,005  anomalies  in  the  northern  1.25  acres  of  the  man- 
portable  area.  Cued  data  collection  was  conducted  for  the  remaining  1 ,429  anomalies  previously 
identified  from  an  EM61-MK2  cart  survey  conducted  in  2012  by  National  Guard  Bureau 
contractor  TetraTech.  This  survey  used  the  MP  System  in  a  litter-carry  configuration. 
Characterization  of  the  system  responses  to  the  Targets  of  Interest  (TOIs)  was  based  on 
previously  acquired  TEMTADS  reference  data.  These  data  were  collected  in  accordance  with 
the  overall  study  objectives  and  demonstration  plan.  Unlike  the  2012  demonstration, 
classification  analysis  of  the  interrogated  anomalies  was  also  conducted.  As  part  of  the 
demonstration,  plastic  pin  flags  were  installed  at  each  target  location  on  the  source  list  prior  to 
data  collection.  Performance  of  the  system  response  was  determined  on  a  twice-daily  basis  using 
the  onsite  IVS.  The  data  segment  (chip)  for  each  anomaly  was  analyzed,  and  dipole  model  fit 
parameters  extracted.  These  results  were  provided  to  the  ESTCP  Program  Office  in  addition  to 
the  archival  data.  Using  these  results,  a  prioritized  dig  list  was  prepared  and  submitted  to  the 
ESTCP  Program  Office  for  scoring. 

5.5.2  Sample  Density 

The  EMI  data  spacing  for  the  TEMTADS  is  fixed  at  40  cm  in  both  directions  by  the  array 
design. 
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5.5,3  Quality  Checks 


Preventative  maintenanee  inspections  were  conducted  at  least  once  a  day  by  all  team  members, 
focusing  particularly  on  the  litter-carry  structure  and  the  cabling.  Any  deficiencies  were 
addressed  according  to  the  severity  of  the  deficiency.  Parts,  tools,  and  materials  for  many 
maintenance  scenarios  were  available  in  the  system  spares  inventory  which  was  located  on  site. 
Status  on  any  break-downs  /  failures  which  would  result  in  long-term  delays  in  operations  would 
have  been  immediately  reported  to  the  ESTCP  Program  Office.  None  occurred. 

The  GPS  data  QC  procedures  and  checks  were  as  follows.  The  status  of  the  RTK  GPS  system 
was  visually  determined  by  the  operator  prior  to  starting  the  data  collection  cycle,  assuring  that 
the  position  information  were  valid  during  the  collection  period,  with  Fix  Quality  (FQ)  3  for  the 
GGK  NMEA  sentence.  A  Fix  Quality  (FQ)  value  of  3  (RTK  Fixed)  is  the  best  accuracy 
(typically  3-5  cm  or  better).  A  FQ  value  of  2  (RTK  Float)  indicates  that  the  highest  level  of  RTK 
has  not  be  reached  yet  and  location  accuracy  can  be  degraded  to  as  poor  as  '-I  m.  FQs  1  &  4 
correspond  to  the  Autonomous  and  DGPS  operational  modes,  respectively.  Data  collected  under 
FQ  3  and  FQ  2  (at  the  discretion  of  the  data  analyst)  are  retained.  The  GGA  NMEA  sentence 
output  from  the  receiver  was  used  for  this  demonstration,  and  the  FQ  values  are  different  {i.e.  FQ 
4  =  Fixed),  but  the  process  is  the  same. 

Two  data  quality  checks  were  performed  on  the  cued  EMI  data.  After  background  subtraction, 
plots  were  made  of  the  48  transmitter  /  receiver  pairs.  An  example  plot  is  shown  in  Figure  5-6 
for  a  horizontal  3”  diameter  x  12”  long  solid  steel  cylinder  at  a  depth  of  45  cm  below  the  sensor 
array.  The  plots  were  visually  inspected  to  verify  that  there  was  a  well-defined  anomaly  without 
extraneous  signals  or  dropouts.  QC  on  the  transmit/receive  cross  terms  was  based  on  the  dipole 
inversion  results.  Our  experience  has  been  that  data  glitches  are  observed  as  reduced  dipole  fit 
coherence. 

In  cued  mode,  the  operator  had  access  to  a  series  of  monostatic  decay  plots  to  allow  for  on-the- 
fly  data  QC.  An  example  monostatic  decay  plot  for  a  high  SNR  anomaly  centered  under  the 
array  is  shown  in  Figure  5-7. 
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Figure  5-6  -  QC  Plot  for  a  3”  x  12”  solid  steel  cylinder,  horizontal  at  a  depth  of  45cm  below  the  sensors. 
The  z,y,x-components  in  each  subplot  are  shown  in  blue,  green,  and  red,  respectively. 


Figure  5-7  -  Screenshot  of  the  Operator  Tablet  Decay  Transient  Display.  A 
large  object  centered  under  the  array  is  indicated. 
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Any  data  set  deemed  unsatisfaetory  by  the  data  analyst  was  flagged  and  not  proeessed  further. 
The  anomaly  eorresponding  to  the  flagged  data  was  logged  for  future  re-aequisition.  Data  whieh 
met  these  standards  were  of  the  quality  typieal  of  the  TEMTADS  system. 

5,5,4  Data  Handling 

Data  were  stored  eleetronieally  as  eolleeted  on  the  baekpaek  data  aequisition  eomputer  hard 
drive.  At  least  onee  daily,  the  data  were  transferred  to  the  off-site  data  analyst  for  QC/analysis. 
Raw  data  and  analysis  results  are  backed  up  from  the  data  analyst’s  computer  to  external  hard 
disks  daily.  These  results  are  archived  on  an  internal  file  server  at  NRL  at  the  end  of  the  survey. 
Examples  of  the  TEMTADS  file  formats  are  provided  in  Appendix  C.  All  field  notes  /  activity 
logs  were  written  in  ink  and  stored  in  archival  laboratory  notebooks.  These  notebooks  were 
archived  at  NRL  or  NAEVA.  Relevant  sections  were  reproduced  in  documents  such  as  this 
demonstration  data  report.  Dr.  Daniel  Steinhurst  is  the  Point  of  Contact  (POC)  for  obtaining  data 
and  other  information.  His  contact  information  is  provided  in  Appendix  B  of  this  report. 

5,6  VALIDATION 

Typically,  at  the  conclusion  of  an  ESTCP  Project  Live  Site  Demonstration,  some  or  all 
anomalies  would  be  excavated.  Each  item  encountered  would  be  identified,  photographed,  its 
depth  below  grade  measured,  its  location  determined  using  cm-level  GPS,  and  the  item  removed 
if  possible.  This  ground  truth  information  would  then  be  used  to  validate  the  objectives  listed  in 
Section  3.0.  Since  the  development  of  the  Demonstration  Plan  Supplement  and  the  Objectives 
laid  out  within,  events  have  transpired  such  that  dig  results  may  not  become  available  to  allow 
for  scoring  the  classification  results  presented  in  this  report. 

6.0  DATA  ANALYSIS  PLAN 

6,1  PREPROCESSING 

The  MP  System  has  four  sensor  elements,  each  comprised  of  a  transmitter  coil  and  a  tri-axial 
receiver  cube.  Eor  each  transmit  pulse,  the  responses  at  all  of  the  receivers  are  recorded.  This 
results  in  48  possible  transmitter  /  receiver  combinations  in  the  data  set  (4  transmitters  x  4 
receiver  cubes  x  3  receiver  axes).  Although  the  data  acquisition  system  records  the  signal  over 
122  logarithmically-spaced  time  gates  starting  at  25  qs,  the  measured  responses  over  the  first  17 
gates  included  distortions  due  to  transmitter  ringing  and  related  artifacts  and  are  discarded.  We 
further  subtract  0.028  ms  from  the  nominal  gate  times  to  account  for  time  delay  due  to  effects  of 
the  receive  coil  and  electronics  [15].  The  delay  was  determined  empirically  by  comparing 
measured  responses  for  test  spheres  with  theory.  This  leaves  105  gates  spaced  logarithmically 
between  0.089  ms  and  24.35  ms.  In  preprocessing,  the  recorded  signals  are  normalized  by  the 
peak  transmitter  current  to  account  for  any  variation  in  the  transmitter  output.  On  average,  the 
peak  transmitter  current  is  approximately  6.5  Amps. 
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The  background  response  is  subtracted  from  each  target  measurement  using  data  collected  at  a 
nearby  target-free  background  location.  The  background  measurements  are  reviewed  for 
variability  and  to  identify  outliers,  which  may  correspond  to  measurements  over  targets.  In 
previous  testing  at  our  Blossom  Point  test  field  and  during  other  demonstrations,  significant 
background  variability  was  not  observed.  It  has  been  possible  to  use  blank  ground  measurements 
from  100  meters  away  for  background  subtraction.  Changes  in  moisture  content  and  outside 
temperature  have  been  shown  to  cause  variation  in  the  backgrounds,  necessitating  care  when 
collecting  data  after  weather  events  such  as  rain. 

6.2  PARAMETER  ESTIMATION 

The  raw  signature  data  from  the  TEMTADS  sensors  reflect  details  of  the  sensor/target  geometry 
as  well  as  inherent  EMI  response  characteristics  of  the  targets  themselves.  In  order  to  separate 
out  the  intrinsic  target  response  properties  from  sensor/target  geometry  effects,  we  inverted  the 
signature  data  to  estimate  principal  axis  magnetic  polarizabilities  for  the  targets.  The 
TEMTADS  data  were  inverted  using  the  standard  induced  dipole  response  model  wherein  the 
effect  of  eddy  currents  set  up  in  the  target  by  the  primary  field  is  represented  by  a  set  of  three 
orthogonal  magnetic  dipoles  at  the  target  location  [16].  The  measured  signal  is  a  linear  function 
of  the  induced  dipole  moment  m,  which  can  be  expressed  in  terms  of  a  time  dependent 
polarizability  tensor  B  as 


m  =  UBUTHo 

where  U  is  the  transformation  matrix  between  the  physical  coordinate  directions  and  the 
principal  axes  of  the  target  and  Ho  is  the  primary  field  strength  at  the  target.  The  eigenvalues  Pi(t) 
of  the  polarizability  tensor  are  the  principal  axis  polarizabilities. 

Given  a  set  of  measurements  of  the  target  response  with  varying  geometries  or  "look  angles"  at 
the  target,  the  data  can  be  inverted  to  determine  the  local  (X,Y,Z)  location  of  the  target,  the 
orientation  of  its  principal  axes  ((t),0,\|/),  and  the  principal  axis  polarizabilities  (Pi,P2,P3).  The 
basic  idea  is  to  search  out  the  set  of  nine  parameters  (X,Y,Z,(t),0,\|/,Pi,P2,P3)  that  minimizes  the 
difference  between  the  measured  responses  and  those  calculated  using  the  dipole  response 
model.  The  location  and  orientation  information  recorded  from  the  GPS  and  IMU  are  then  used 
to  locate  the  fit  results  in  three-dimensional  space. 

Eor  TEMTADS  data,  inversion  is  accomplished  by  a  two-stage  method.  In  the  first  stage,  the 
target’s  (X,Y,Z)  dipole  location  beneath  are  solved  for  non-linearly.  At  each  iteration  within  this 
inversion,  the  nine  element  polarizability  tensor  (B)  is  solved  linearly.  We  require  that  this 
tensor  be  symmetric;  therefore,  only  six  elements  are  unique.  Initial  guesses  for  X  and  Y  are 
determined  by  a  signal-weighted  mean.  The  routine  normally  loops  over  a  number  of  initial 
guesses  in  Z,  keeping  the  result  giving  the  best  fit  as  measured  by  the  chi-squared  value.  The 
non-linear  inversion  is  done  simultaneously  over  all  time  gates,  such  that  the  dipole  (X,Y,Z) 
location  applies  to  all  decay  times.  At  each  time  gate,  the  eigenvalues  and  angles  are  extracted 
from  the  polarizability  tensor. 
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In  the  second  stage,  six  parameters  are  used;  the  three  spatial  parameters  (X,Y,Z)  and  three 
angles  representing  the  yaw,  pitch,  and  roll  of  the  target  (Euler  angles  (t),0,xi/).  Here  the 
eigenvalues  of  the  polarizability  tensor  are  solved  for  linearly  within  the  6-parameter  non-linear 
inversion.  In  this  second  stage  both  the  target  location  and  its  orientation  are  required  to  remain 
constant  over  all  time  gates.  The  value  of  the  best  fit  X,Y,Z  from  the  first  stage,  and  the  median 
value  of  the  first-stage  angles  are  used  as  an  initial  guess  for  this  stage.  Additional  loops  over 
depth  and  angles  are  included  to  better  ensure  finding  the  global  minimum. 

Figure  6-1  shows  an  example  of  the  principal  axis  polarizabilities  determined  from  TEMTADS 
array  data.  The  target,  a  mortar  fragment,  is  a  slightly  bent  plate  about  V2  cm  thick,  25  cm  long, 
and  15  cm  wide.  The  red  curve  is  the  polarizability  when  the  primary  field  is  normal  to  the 
surface  of  the  plate,  while  the  green  and  blue  curves  correspond  to  cases  where  the  primary  field 
is  aligned  along  each  of  the  edges. 

Not  every  target  on  the  target  list  had  a  strong  enough  TEM  response  to  support  extraction  of 
target  polarizabilities.  All  of  the  data  were  run  through  the  inversion  routines,  and  the  results 
were  manually  screened  to  identify  those  targets  that  could  not  be  reliably  parameterized. 
Several  criteria  were  used  in  this  process:  signal  strength  relative  to  background,  dipole  fit  error 
(difference  between  data  and  model  fit  to  data),  and  the  visual  appearance  of  the  polarizability 
curves. 


Figure  6-1  -  Principal  axis  polarizabilities  for  a  0.5  cm  thick 
by  25cm  long  by  15cm  wide  mortar  fragment. 
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6.3  CLASSIFIER  AND  TRAINING 


Target  classification  was  based  on  a  library  matching  procedure  wherein  we  compared  the  results 
of  a  dipole  inversion  of  the  TEM  array  data  to  principal  axis  polarizabilities  drawn  from  a  library 
of  known  signatures.  The  match  to  a  library  entry  is  based  on  three  criteria:  the  amplitude  of  the 
primary  polarizability,  and  the  ratio  of  the  second  and  third  polarizabilities  to  the  first.  Multiple 
metrics,  each  of  which  run  from  0  (terrible  match)  to  1  (perfect  match),  are  used  to  evaluate  each 
match. 

Our  experience  with  these  sensors  has  been  that  principal  polarizabilities  determined  from  in-air 
measurements  are  indistinguishable  from  those  determined  from  measurements  taken  over  buried 
targets.  We  have  an  extensive  collection  of  inert  military  munitions  collected  from  many  sources 
which  were  measured  at  our  home  facility  using  the  TEMTADS  family  of  sensors  mounted  on  a 
test  stand.  We  have  also  assembled  a  fairly  extensive  polarizability  database  for  clutter  items 
recovered  from  several  different  live  sites.  These  data  collections  were  used  as  training  data  for 
establishing  UXO/clutter  discrimination  boundaries  on  the  library  match  metrics. 

6.4  DATA  PRODUCT  SPECIFICATIONS 

For  all  anomalies  and  IVS  measurements,  the  deliverables  for  this  demonstration  were  the  raw 
data  files,  background-subtracted  data  files,  and  fit  results.  A  prioritized  dig  list  for  all  anomalies 
was  also  provided.  All  deliverables  were  submitted  to  the  ESTCP  Program  Office.  See 
Appendix  C  for  the  detailed  data  product  specifications. 

7.0  PERFORMANCE  RESULTS 

The  performance  objectives  for  the  demonstration  were  given  in  Table  3-1  and  are  repeated  here 
in  Table  7-1  and  Table  7-4.  The  results  for  each  criterion  are  subsequently  discussed  in  the 
following  sections.  Since  the  development  of  the  Demonstration  Plan  Supplement  and  the 
Objectives  laid  out  within,  events  have  transpired  such  that  dig  results  may  not  become  available 
to  allow  for  scoring  the  classification  results  presented  in  this  report.  As  such,  the  requisite  data 
for  evaluating  the  demonstration  performance,  so  noted  individually,  may  not  become  available 
at  a  future  point. 
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Table  7-1  -  Data  Collection  Performance  Results  for  this  Demonstration 


Performance 

Objective 

Metric 

Data  Required 

Success  Criteria 

Success? 

(Yes/No) 

Data  Collection  Objectives 

Instrument 

Verification 

Strip  (P/S) 
Results 

System  responds 
consistently  to 
emplaced  items 

Daily  IVS  data 

<  10%  RMS 
variation  in  P 
amplitudes  and  in 
fit  location 

Yes 

Cued 

interrogation  of 
anomalies 

Instrument 

position 

Cued  Data 

100%  of 
anomalies  where 
the  center  of  the 
instrument  is 
positioned  within 
40  cm  of  actual 
target  location 

Yes 

7.1  OBJECTIVE:  REPEATABILTY  OF  INSTRUMENT  VERIFICATION  STRIP 
MEASUREMENTS 

This  objective  verifies  that  the  sensor  system  is  in  good  working  order  and  is  collecting 
physically  valid  data  each  day.  The  items  emplaced  in  the  Instrument  Verification  Strip  (IVS) 
were  surveyed  twice  daily.  The  amplitude  of  the  derived  response  coefficients  and  fit  location 
for  each  emplaced  item  were  compared  to  the  running  average  of  the  demonstration  for 
reproducibility. 

7.1.1  Metric 

The  reproducibility  of  the  measured  response  of  the  sensor  system  to  the  emplaced  items  defines 
this  metric. 

7.1.2  Data  Requirements 

The  tabulated  fit  parameters  for  the  data  corresponding  to  each  emplaced  item  in  terms  of 
derived  response  coefficients  and  fit  location. 

7.1.3  Success  Criteria 

The  objective  was  considered  met  if  the  root-mean-squared  (RMS)  amplitude  variation  of  the 
derived  response  coefficients  and  fit  location  were  less  than  10%. 
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7.1,4  Results 


This  objective  is  considered  a  success.  Each  emplaced  item  in  the  IVS  was  measured  twice 
daily,  once  before  starting  the  data  collection  process  and  a  second  time  before  shutting  the 
system  down  at  the  end  of  each  day.  Details  of  the  contents  of  the  CIA  IVS  are  given  in  Table 
5-1.  A  background  spot  was  not  provided  in  the  IVS.  One  was  found  nearby  using  the  MP 
System  in  concert  with  a  handheld  magnetometer. 

All  data  sets  for  each  of  the  emplaced  IVS  items  were  inverted  using  the  data  analysis 
methodology  discussed  in  Section  6.0,  and  the  estimated  target  parameters  determined. 
Geolocation  and  sensor  platform  orientation  are  now  available  for  the  MP  System  in  cued  mode, 
unlike  previous  demonstrations  of  the  MP  System. 

The  results  for  the  seventeen  cued  mode  IVS  measurements  are  given  in  Table  7-2  and  shown  in 
Figure  7-1.  The  RMS  variation  in  the  magnetic  polarizability  amplitudes  at  0.089  ms  were  less 
than  4%  for  all  three  magnetic  polarizabilities.  The  aggregate  position  and  depth  error  statistics 
for  the  IVS  items  are  listed  in  Table  7-3  and  shown  in  Figure  7-2.  The  position  error  is  defined 
as  the  fit  position  (or,  equivalently,  the  inverted  position  parameter)  minus  the  position  given  in 
Table  5-1.  The  RMS  variation  in  the  position  errors  for  each  emplaced  IVS  item  was  under  2  cm. 
Depth  error  is  expressed  as  the  difference  between  the  fitted  depth  and  the  listed  emplacement 
depth. 

The  RMS  variation  in  the  depth  errors  for  each  emplaced  IVS  item  was  0.23  cm  (1%)  or  less. 
The  location  and  depth  of  the  IVS  items  could  not  be  independently  verified  by  our  team.  In  the 
case  of  the  depth,  the  value  is  calculated  from  several  provided  pieces  of  information  as  noted  in 
Table  5-1.  Discussion  of  any  mean  offsets  would  require  detailed  recovery  of  the  items. 

Table  7-2  -Summary  of  the  Amplitude  Variations  at  0.089  ms  in  the  Derived  Response  Coefficients  for 
All  Items  Emplaced  in  the  IVS. 


Item 

Pi  Amplitude  (mfi 

1 

32  Amplitude  (m^ 

P3  Amplitude  (m^ 

Min 

Max 

Mean 

RMS 

Min 

Max 

Mean 

RMS 

Min 

Max 

Mean 

RMS 

155mmP 

58.04 

64.28 

60.89 

1.730 

55.33 

58.99 

57.54 

1.042 

48.95 

53.08 

51.73 

1.110 

105mmP 

23.74 

26.59 

24.84 

0.779 

19.89 

21.37 

20.60 

0.379 

18.59 

20.07 

19.25 

0.386 

Med.  ISO  #I 

8.56 

9.08 

8.74 

0.152 

5.13 

5.48 

5.34 

0.097 

4.67 

4.87 

4.76 

0.056 

Med.  ISO  #2 

9.32 

10.36 

9.82 

0.299 

5.54 

6.25 

5.83 

0.224 

5.11 

5.41 

5.25 

0.097 

Table  7-3  -  Summary  of  Position  and  Depth  Error  Statistics  for  all  items  emplaced  in  the  IVS. 


Item 

Easting  Position  Error  (cm) 

Northing  Position  Error  (cm) 

Depth  Error  (cm) 

Min 

Max 

Mean 

RMS 

Min 

Max 

Mean 

RMS 

Min 

Max 

Mean 

RMS 

155mniP 

1.84 

9.05 

5.41 

1.88 

-3.40 

2.10 

-0.29 

1.317 

-8.52 

-7.97 

-8.13 

0.15 

105mmP 

1.45 

-1.57 

1.67 

-4.50 

1.10 

-1.46 

1.396 

-7.19 

-6.62 

-6.86 

0.18 

Med.  ISO  #1 

3.69 

-0.04 

1.30 

-4.40 

2.40 

0.04 

1.692 

-6.79 

-5.97 

-6.25 

0.20 

Med.  ISO  #2 

1.32 

6.72 

4.62 

1.18 

-5.70 

-0.10 

-3.31 

1.46 

-7.69 

-6.96 

-7.26 

0.23 
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Table  7-4  -  Analysis  and  Classification  Performance  Results  for  this  Demonstration 


Performance 

Objective 

Metric 

Data  Required 

Success  Criteria 

Success? 

(Yes/No) 

Analysis  and  Classification  Objectives 

Maximize 

correct 

classification  of 
TOI 

Number  of  TOI 
retained 

Ranked  anomaly  lists 

Results  of  intrusive 
investigation 

Approach 
correctly 
classifies  all  TOI 

Not 

Evaluated 

Maximize 

correct 

classification  of 
non-TOI 

Number  of  false 
alarms  eliminated 

Ranked  anomaly  lists 

Results  of  intrusive 
investigation 

Reduction  of 
clutter  digs 
required  by  >75% 
while  retaining  all 
TOI 

Not 

Evaluated 

Specification  of 
no-dig  threshold 

Probability  of 
correct 

classification  of 
TOI  and  number 
of  false  alarms  at 
operating  point 

Specified  threshold 

Results  of  intrusive 
investigation 

Threshold 
specified  to 
achieve  criteria 
above 

Not 

Evaluated 

Minimize 
number  of 
anomalies  that 
cannot  be 
analyzed 

Number  of 
anomalies  that 
must  be  classified 
as  “Unable  to 
Analyze” 

Extracted  target 
parameters 

Reliable  target 
parameters  can  be 
estimated  for  > 

95%  of  anomalies 

Yes 

Correct 
estimation  of 
target 
parameters 

Accuracy  of 
estimated  target 
parameters  for 
seed  items 

Extracted  target 
parameters 

Results  of  intrusive 
investigation 

Polarizabilities  ± 
20% 

X,  Y<  15  cm 
(1g) 

Z  <  10  cm  (1g) 

Not 

Evaluated 
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Figure  7-1  -  Amplitude  variations  at  0.089  ms  in  the  derived  response 
coefficients  for  all  items  emplaced  in  the  IVS.  Pi  is  in  red;  P2  is  in  green;  and  Ps 
is  in  blue. 


•  Northing 

•  Easting 


Figure  7-2  -  Position  error  statistics  for  the  four  items  emplaced  in  the  IVS  (left  panel).  Easting  data  are 
in  black  and  Northing  data  are  in  red.  Depth  error  statistics  for  the  same  items  (right  panel). 

7.2  OBJECTIVE:  CUED  INTERROGATION  OF  ANOMALIES 

To  collect  EMI  data  of  the  highest  quality  for  UXO/clutter  classification,  the  anomaly  must  be 
illuminated  along  its  three  principal  axes.  To  ensure  this,  the  data  collection  pattern  (in  this  case 
the  TEMTADS  array)  must  be  positioned  directly  over  the  center  of  the  anomaly. 
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7.2,1  Metric 


The  metric  for  this  objective  is  the  percentage  of  anomalies  that  were  within  the  acceptable 
distance  of  the  center  of  the  instrument  during  data  collection  from  the  actual  target  location. 

7.2.2  Data  Requirements 

After  preliminary  data  collection  and  any  reacquisition  cycles  were  complete,  the  offset  from 
array  center  to  inverted  target  location  were  determined  for  each  anomaly.  The  offset  distance 
was  required  to  be  less  than  40  cm. 

7.2.3  Success  Criteria 

The  objective  was  considered  met  if  the  center  of  the  instrument  was  positioned  within  40  cm  of 
the  anomaly  fit  location  for  100%  of  the  cued  anomalies.  Exceptions  were  made  for  anomalies 
which  were  partially  or  completely  occluded  by  obstructions  such  as  trees  or  were  located  in 
areas  where  intrusive  investigation  were  not  planned  (e.g.  roads).  Several  earthen  berms  formed 
from  bulldozed  dirt,  vegetation,  and  MEC  were  also  excluded  from  the  demonstration  for  not 
being  good  candidates  for  classification.  Anomalies  located  within  a  known  test  plot  were  also 
excluded  at  the  request  of  site  personnel. 

7.2.4  Results 

A  finalized  dig  list,  including  fit  results  from  both  single-  and  multi-dipole  solvers,  was  available 
as  of  August  26,  2013.  Based  on  the  finalized  fit  results,  the  offset  from  the  array  center  to  the 
fit  location  of  each  classified  target  was  calculated.  All  offsets  were  less  than  or  equal  to  40  cm 
except  for  three  targets.  Eor  flag  CE-3580,  the  fit  result  source  was  the  same  location  as  the 
result  for  nearby  flag  CE-3569.  This  was  interpreted  to  indicate  no  source  at  the  position  of  flag 
CE-3580.  The  remaining  two  flags,  CE-4009  and  CE-4101  appeared  to  be  from  a  large  source 
located  outside  the  survey  area  boundary  and  were  classified  as  “cannot  analyze.”  This 
Objective  is  considered  successful. 

7.3  OBJECTIVE:  MAXIMIZE  CORRECT  CLASSIFICATION  OF  TOI 

This  is  one  of  the  two  primary  measures  of  the  effectiveness  of  the  classification  approach.  By 
collecting  high-quality  data  and  analyzing  those  data  with  advanced  parameter  estimation  and 
classification  algorithms,  we  should  be  able  to  classify  the  targets  with  high  efficiency.  This 
objective  concerns  the  component  of  the  classification  problem  that  involves  correct 
classification  of  TOI.  Since  the  development  of  this  objective  as  part  of  the  Demonstration  Plan 
Supplement,  events  have  transpired  such  that  dig  results  may  not  become  available  to  allow  for 
scoring  the  classification  results  presented  in  this  report.  As  such,  the  requisite  data  for 
evaluating  this  objective  may  not  become  available  at  a  future  point. 
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7,3.1  Metric 


The  metric  for  this  objective  is  the  number  of  items  on  the  MP  System  anomaly  list  that  were 
correctly  classified  as  TOI. 

7.3.2  Data  Requirements 

A  ranked  anomaly  list  was  prepared  for  the  targets  on  the  MP  System  anomaly  list. 

7.3.3  Success  Criteria 

For  the  purposes  of  this  demonstration,  the  objective  was  considered  met  if  all  of  the  TOI  were 
correctly  labeled  as  TOI  on  the  ranked  anomaly  list.  This  is  a  more  stringent  criterion  than 
required  for  the  site  clean-up  objectives  listed  in  the  Source  Removal  Work  Plan.  It  was  possible 
to  achieve  results  adequate  for  the  site-specific  cleanup  objectives  but  below  the  100%  correct 
classification  specified. 

7.3.4  Results 

The  data  for  determining  the  success  of  this  objective  are  not  currently  available,  and  may  not 
become  available  in  the  future.  If  the  data  become  available,  the  analysis  can  be  revisited. 

7.4  OBJECTIVE:  MAXIMIZE  CORRECT  CLASSIFICATION  OF  NON-TOI 

This  is  the  second  of  the  two  primary  measures  of  the  effectiveness  of  the  classification 
approach.  By  collecting  high-quality  data  and  analyzing  those  data  with  advanced  parameter 
estimation  and  classification  algorithms  we  expect  to  be  able  to  classify  the  targets  with  high 
efficiency.  This  objective  concerns  the  component  of  the  classification  problem  that  involves 
false  alarm  reduction.  Since  the  development  of  this  objective  as  part  of  the  Demonstration  Plan 
Supplement,  events  have  transpired  such  that  dig  results  may  not  become  available  to  allow  for 
scoring  the  classification  results  presented  in  this  report.  As  such,  the  requisite  data  for 
evaluating  this  objective  may  not  become  available  at  a  future  point. 

7.4.1  Metric 

The  metric  for  this  objective  is  the  number  of  items  on  the  MP  System  anomaly  list  that  were 
correctly  classified  as  non-TOI. 

7.4.2  Data  Requirements 

A  ranked  anomaly  list  was  prepared  for  the  targets  on  the  MP  System  anomaly  list. 

7.4.3  Success  Criteria 

The  objective  was  considered  met  if  more  than  75%  of  the  non-TOI  items  were  correctly  labeled 
as  non-TOI  while  retaining  all  of  the  TOI  on  the  dig  list. 
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7.4,4  Results 


The  data  for  determining  the  suceess  of  this  objeetive  are  not  currently  available,  and  may  not 
become  available  in  the  future.  If  the  data  become  available,  the  analysis  can  be  revisited. 

7.5  OBJECTIVE:  SPECIFICATION  OF  NO-DIG  THRESHOLD 

In  a  retrospective  analysis  as  was  intended  for  this  demonstration,  it  is  possible  to  tell  the  true 
classification  capabilities  of  a  classification  procedure  based  solely  on  the  ranked  anomaly  list 
submitted.  In  a  real-world  scenario,  all  targets  may  not  be  dug  so  the  success  of  the  approach  will 
depend  on  the  ability  of  an  analyst  to  accurately  specify  their  dig/no-dig  threshold.  Since  the 
development  of  this  objective  as  part  of  the  Demonstration  Plan  Supplement,  events  have 
transpired  such  that  dig  results  may  not  become  available  to  allow  for  scoring  the  classification 
results  presented  in  this  report.  As  such,  the  requisite  data  for  evaluating  this  objective  may  not 
become  available  at  a  future  point. 

7.5.1  Metric 

The  probability  of  correct  classification  of  TOI,  Pciass,  and  number  of  false  alarms,  Nfa,  at  the 
demonstrator-specified  threshold  were  the  metrics  for  this  objective. 

7.5.2  Data  Requirements 

A  ranked  anomaly  list  with  a  dig/no-dig  threshold  indicated  was  prepared  for  the  MP  System 
anomaly  list. 

7.5.3  Success  Criteria 

The  objective  was  considered  met  if  more  than  75%  of  the  non-TOI  items  were  correctly  labeled 
as  non-TOI  while  retaining  all  of  the  TOI  at  the  demonstrator-specified  threshold. 

7.5.4  Results 

The  data  for  determining  the  success  of  this  objective  are  not  currently  available,  and  may  not 
become  available  in  the  future.  If  the  data  become  available,  the  analysis  can  be  revisited. 

7.6  OBJECTIVE:  MINIMIZE  NUMBER  OF  ANOMALIES  THAT  CANNOT  BE 
ANALYZED 

Evaluation  of  anomalies  for  which  reliable  parameters  could  not  be  estimated  cannot  continue  in 
the  classifier  analysis  pipeline.  These  anomalies  must  be  placed  in  the  dig  category  and  reduce 
the  effectiveness  of  the  classification  process. 
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7,6,1  Metric 


The  number  of  anomalies  for  whieh  reliable  parameters  could  not  be  estimated  was  the  metric 
for  this  objective. 

7.6.2  Data  Requirements 

A  list  of  all  extracted  target  parameters  was  provided  as  part  of  the  results  submission  along  with 
a  list  of  those  anomalies  for  which  parameters  could  not  be  reliably  estimated. 

7.6.3  Success  Criteria 

The  objective  was  considered  met  if  reliable  parameters  were  estimated  for  >  95%  of  the 
anomalies  on  the  anomaly  list. 

7.6.4  Results 

This  Objective  is  considered  successful.  Of  the  1,429  remaining  flag  locations,  1,291  were 
accessible  for  measurement  and  not  located  in  areas  that  were  considered  ‘out-of-bounds,’  such 
as  the  roads  and  a  test  area  located  within  the  survey  area.  Of  the  1,291  flags  investigated,  data 
were  collected  from  which  reliable  fit  parameters  could  be  extracted  for  all  but  five  flag  locations 
(99.4%). 

7,7  OBJECTIVE:  CORRECT  ESTIMATION  OF  TARGET  PARAMETERS 

This  objective  involves  the  accuracy  of  the  target  parameters  that  were  estimated  in  the  first 
phase  of  the  analysis.  Successful  classification  is  only  possible  if  the  input  features  are  internally 
consistent.  The  obvious  way  to  satisfy  this  condition  is  to  estimate  the  various  target  parameters 
accurately. 

7.7.1  Metric 

Accuracy  of  estimation  of  target  parameters  was  the  metric  for  this  objective. 

7.7.2  Data  Requirements 

The  estimated  parameters  for  the  seed  items  and  the  ground  truth  for  the  seed  items  are  required 
data. 

7.7.3  Success  Criteria 

The  objective  was  considered  met  if  the  estimated  polarizabilities  were  within  ±  20%,  the 
estimated  X,  Y  locations  were  within  15  cm  (la),  and  the  estimated  depths  were  within  10  cm 
(la). 
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7,7,4  Results 


The  data  for  determining  the  success  of  this  objective  are  not  currently  available,  and  may  not 
become  available  in  the  future.  If  the  data  become  available,  the  analysis  can  be  revisited. 

7,8  BACKGROUND  SIGNAL  VARIATION  RESULTS 

A  group  of  anomaly-free  areas  along  the  road  bisecting  the  ManPortable  subarea  were  identified 
in  advance  from  the  EM61-MK2  data  set  and  by  inspection  with  a  hand-held  magnetometer.  An 
example  of  a  background  measurement  being  made  is  shown  in  Figure  7-3.  Each  background 
location  was  confirmed  to  be  anomaly-free  prior  to  prolonged  use  with  the  MP  System.  Any 
location  found  to  exhibit  an  anomaly  was  discarded  and  not  used  further.  Since  the  viable 
locations  all  provided  roughly  comparable  responses,  a  convenient  subset  of  the  locations  was 
chosen  to  be  visited  periodically  throughout  each  day  of  the  demonstration.  All  67  field 
background  measurements  taken  for  the  duration  of  the  survey  are  shown  in  Figure  7-4,  and  are 
presented  as  the  mean  and  standard  deviation  of  the  four  monostatic  measured  signals.  Dates  are 
presented  as  Julian  dates,  or  the  day  of  the  year.  July  15,  2013  is  Julian  date  196.  Table  7-5 
tabulates  the  intraday  variations  of  the  mean  and  standard  deviation  quantities  from  Figure  7-4. 
Additional  background  measurements  were  made  near  the  IVS  but  are  not  included  here. 


Figure  7-3  -  Team  Members  Preparing  for  a  Background  Measurement 
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Table  7-5  -  Summary  of  the  Daily  Variation  in  the  Mean  and  Standard  Deviation  of  the  Signals  Measured 
for  the  MP  System  Background  Areas. 


Date 

#of 

Bkgs. 

Mean  Z 
(mV/Amp) 

Std.  Dev.  Z 
(mV/Amp) 

Mean  Y 
(mV/Amp) 

Std.  Dev.  Y 
(mV/Amp) 

Mean  X 
(mV/Amp) 

Std.  Dev.  X 
(mV/A) 

7/15/2013 

4 

29.61 

0.86 

1.08 

0.58 

1.41 

0.51 

7/16/2013 

6 

29.66 

0.60 

1.29 

0.45 

1.53 

0.49 

7/17/2013 

9 

28.03 

1.09 

1.13 

0.52 

1.54 

0.64 

7/18/2013 

12 

29.72 

0.48 

1.00 

0.42 

1.38 

0.48 

7/20/2013 

10 

28.10 

0.66 

1.04 

0.70 

1.32 

0.37 

7/21/2013 

8 

28.09 

0.59 

0.97 

0.57 

1.42 

0.46 

7/22/2013 

10 

27.19 

0.80 

0.99 

0.44 

1.21 

0.55 

7/23/2013 

4 

27.04 

0.57 

0.82 

0.42 

1.18 

0.48 

7/24/2013 

4 

26.87 

0.87 

1.12 

0.62 

1.35 

0.53 
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Figure  7-4  -  Intra-  and  inter-  daily  variations  in  the  response  of  the  MP  System  to  background  anomaly- 
free  areas  through  the  duration  of  the  demonstration.  The  upper  panel  plots  the  average  measured  signal 
of  the  four  monostatic,  Z-axis  quantities  at  0.089  ms,  while  the  bars  represent  the  standard  deviation  of 
those  quantities  (i.e.  la  about  the  mean).  The  red  and  green  points  in  the  lower  panel  plot  the  average 
measured  signal  of  the  four  monostatic,  X-  and  Y-axis  quantities  at  0.089  ms,  respectively. 
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8.0  COST  ASSESSMENT 


8.1  COST  MODEL 

The  cost  elements  tracked  for  this  demonstration  are  detailed  in  Table  8-1.  The  provided  cost 
elements  are  based  on  a  model  developed  for  cost  estimation  for  the  MP  System  at  Camp  Beale 
in  2011  [6].  The  model  assumes  a  two-person  field  crew  and  one  data  analyst.  For  this  site  a 
third  person  was  required  due  to  the  litter-mode  deployment.  While  the  MP  system  is  not 
currently  commercially  available,  an  estimated  daily  rental  rate  is  provided  for  comparison  to 
other  technologies.  The  rental  rate  is  based,  in  part,  on  the  costs  of  items  purchased  in  prototype 
quantities  (single  units)  and  would  presumably  decrease  significantly  if  the  items  were  procured 
at  production  quantity  levels. 

8.2  COST  DRIVERS 

Two  factors  were  expected  to  be  strong  drivers  of  cost  for  this  technology  as  demonstrated.  The 
first  is  the  number  of  anomalies  which  can  be  surveyed  per  day.  Higher  productivity  in  data 
collection  equates  to  more  anomalies  investigated  for  a  given  period  of  time  in  the  field.  The 
time  required  for  analyzing  individual  anomalies  can  be  significantly  higher  than  for  other,  more 
traditional  methods  and  could  become  a  cost  driver  due  to  the  time  involvement.  The  thoughtful 
use  of  available  automation  techniques  for  individual  anomaly  analysis  with  operator  QC  support 
can  moderate  this  effect. 

In  the  cost  model  presented  in  Table  8-1,  the  data  processing  costs  listed  are  for  data  QC  and  for 
parameter  estimation  using  the  industry-standard  single-dipole  model  solver.  In  a  heavily- 
cluttered  environment  such  as  the  MMR  CIA,  routine  use  of  the  multi-dipole  model  solver,  or 
multi-solver,  was  indicated  and  was  used.  Also  no  specific  cost  element  was  listed  for  the 
classification  process.  As  the  classification  process  is  ongoing,  no  cost  estimate  is  given  at  this 
time.  However,  the  experience  so  far  indicates  that  when  use  of  the  multi-solver  is  required, 
significantly  more  computer  time  is  required  to  model  the  data.  While  more  computer  time  is 
required,  additional  data  analyst  time  is  not  typically  required  during  the  processing. 
Significantly  more  data  analyst  time  is  required  however  to  evaluate  the  results  of  the  multi¬ 
solver  and  to  incorporate  them  into  the  classification  pipeline.  The  cost  model  will  continue  to 
be  updated  for  continued  use  in  the  future  as  additional  cost  data  become  available. 

8.3  COST  BENEFIT 

The  main  benefit  to  using  a  UXO  classification  process  is  cost-related.  The  ability  to  reduce  the 
number  of  non-hazardous  items  that  have  to  be  dug  or  have  to  be  dug  as  presumptively- 
hazardous  items  directly  reduces  the  cost  of  a  remediation  effort.  The  additional  information 
provided  by  these  sensor  systems  significantly  improved  anomaly  classification  performance 
over  traditional  methods.  If  there  is  buy-in  from  the  stakeholders  to  use  these  techniques,  this 
information  can  be  used  to  reduce  costs. 
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Table  8-1  -  TEMTADS  MP  2x2  Cart  Tracked  Costs 


Cost  Element 

Data  Tracked 

Cost 

Data  Collection  Costs 

Pre/Post 

Survey  Activities 

Component  costs  and  integration  costs 
•  Spares  and  repairs 

$3,500 

Cost  to  pack  the  array  and  equipment, 
mobilize  to  the  site,  and  return 

•  Personnel  required  to  pack 

•  Packing  hours 

•  Personnel  to  mobilize 

•  Mobilization  hours 

•  Transportation  costs 

$12,450 

1 

16 

3 

8 

$7,250 

Cost  to  assemble  the  system,  perform 
initial  calibration  tests 

•  Personnel  required 

•  Hours  required 

$780 

3 

2 

Survey  Costs 

Unit  cost  per  anomaly  investigated. 
This  will  be  calculated  as  daily  survey 
costs  divided  by  the  number  of 
anomalies  investigated  per  day. 

•  Equipment  Rental  (day) 

•  Daily  calibration  (hours) 

•  Survey  personnel  required 

•  Survey  hours  per  day 

•  Daily  equipment  break-down  and 
storage  (hours) 

$7,15  /  anom. 

$190 

0.5 

2 

8 

0.5 

Processing  Costs 

$10,85  /  anom. 

Preprocessing 

Time  required  to  perform  standard  data 
clean  up  and  to  merge  the  location  and 
geophysical  data. 

3  min/anomaly 

Parameter 

Estimation 

Time  required  to  extract  parameters  for 
all  anomalies. 

2  min/anomaly 
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9.0  SCHEDULE  OF  ACTIVITIES 


Figure  9-1  gives  the  overall  sehedule  for  the  demonstration  ineluding  deliverables. 


Activity  Name 

2013 

Jun 

Jul 

Aug 

Sept 

MMR  CIA  TEMTADS  MP  Demonstration 

Draft  Demonstration  Plan 

◄ 

► 

TEMTADS  MP  Data  Collection 

Data  Analysis 

Draft  Demonstration  Data  Report 

Jun 

Jul 

Aug 

Sept 

Figure  9-1  -  Schedule  of  all  demonstration  activities  including  deliverables. 

lO.OMANAGEMENT  AND  STAFFING 

The  responsibilities  for  this  demonstration  are  outlined  in  Figure  10-1.  Dan  Steinhurst  (Nova 
Researeh)  was  the  PI  of  this  demonstration.  Dan  Steinhurst  filled  the  roles  of  Site  /  Projeet 
Supervisor.  John  Brezniek  was  the  projeet  lead  for  NAEVA  Geophysies.  Cora  Blits  (NAEVA) 
was  the  Quality  Assuranee  Offieer.  Glenn  Flarbaugh  (Nova  Researeh)  was  the  Site  Safety 
Officer.  His  duties  included  data  collection  and  safety  oversight  for  the  entire  team.  Cora  Blits, 
Alison  Paski,  and  Jon  Guillard  (NAEVA)  served  as  Data  Analysts.  Ben  Dameron  (NAEVA)  and 
Andrew  Benson  (NAEVA)  served  as  Data  Acquisition  Operators. 


Figure  10-1  -  Management  and  Staffing  Wiring  Diagram. 
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APPENDIX  A.  HEALTH  AND  SAFETY  PLAN 


An  abbreviated  Health  and  Safety  Plan  was  generated  for  this  demonstration.  All  emergeney 
information  sueh  as  contact  numbers  and  directions  to  nearby  medical  facilities  are  provided  in 
that  document.  The  contents  are  reproduced  here. 

A.l  DIRECTIONS  TO  FALMOUTH  HOSPITAL 

Directions  to  the  Falmouth  Hospital  in  Falmouth,  MA  are  as  follows,  starting  at  the  main  gate  to 
Camp  Edwards  on  Connery  Avenue.  See  Figure  A-1  for  the  overall  route. 

1)  Head  Northeast  on  Connery  Avenue  for  1 .4  miles. 

2)  At  the  traffic  circle,  take  the  exit  onto  MA-28  South,  drive  for  9.1  miles. 

3)  Turn  Right  onto  Ter  Huen  Drive,  drive  for  0. 1  miles. 

4)  Turn  Left  onto  Bramble  Bush  Drive,  Falmouth  Hospital  is  on  the  Right. 

Falmouth  Hospital  is  located  at  100  Ter  Heun  Drive,  Falmouth,  MA  02540,  (508)  548-5300.  The 
total  distance  to  travel  is  10.6  miles  and  should  take  15  minutes. 
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Figure  A-1  -  Area  map  showing  the  location  of  the  Falmouth  Flospital  with 
respect  to  Camp  Edwards. 
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A.2  EMERGENCY  TELEPHONE  NUMBERS 


Telephone  numbers  for  medieal  fire  and  other  emergeneies  will  be  available  on  site  for  use  by  all 
project  personnel  in  the  event  of  an  emergency  and  are  provided  in  Table  A-1.  All  vehicles  will 
contain  a  cellular  phone  (including  the  phone  list)  to  allow  emergency  communications  in  the 
event  of  an  accident.  The  telephone  area  code  for  this  area  is  508. 

Table  A-1  -  Emergency  Contact  Numbers 


Agency 

Emergency 

Phone 

Number 

Non- 

Emergency 
Phone  Number 

Location 

Bourne  Fire  Department 

911 

(508)  759-4412 

130  Main  Street,  Buzzards  Bay, 
MA  02532 

Cape  Cod  Ambulance 

(508)  833-3928 

15  Jan  Sebastian  Drive 
Sandwich,  MA  02563 

Bourne  Police  Department 

911 

(508)  759-4451 

175  Main  Street,  Buzzards  Bay, 
MA  02532 

Falmouth  Hospital 

(508)  548-5300 

100  Ter  Heun  Drive,  Falmouth, 
MA  02540 

C  vs/pharmacy 

(508)  759-1097 

6  Head  of  the  Bay  Road,  Bourne, 
MA,  02532 

Regional  Center  for  Poison 
Control  and  Prevention 

(800)-222-1222 

http://www.maripoisoncenter.com/ 
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APPENDIX  B.  POINTS  OF  CONTACT 


POINT  OF  CONTACT 

ORGANIZATION 

Phone 

Fax 

e-mail 

Role  in  Project 

Dr.  Anne  Andrews 

ESTCP  Program  Office 

4800  Mark  Center  Drive,  Suite 
17D08  Alexandria,  VA  22350-3605 

571-372-6565  (V) 
571-372-6386  (F) 
anne.andrews@osd.mil 

Acting  Director, 
ESTCP 

Dr.  Herb  Nelson 

ESTCP  Program  Office 

4800  Mark  Center  Drive,  Suite 
17D08  Alexandria,  VA  22350-3605 

571-372-6400  (V) 
571-372-6386  (F) 
202-215-4844  (C) 
herb.nelson@nrl.navy.mil 

Program 
Manager,  MR 

Mr.  Daniel  Reudy 

HydroGeoLogic,  Inc. 

1 1 107  Sunset  Hills  Road,  Suite  400 
Reston,  VA  20190 

703-736-4531  (V) 
dmedy@hgl.com 

Program 
Manager’s 
Assistant,  MR 

Dr.  Dan  Steinhurst 

Nova  Research,  Inc. 

1900  Elkin  St.,  Ste.  230 

Alexandria,  VA  22308 

202-767-3556  (V) 
202-404-8119  (F) 
703-850-5217  (C) 
dan.steinhurst@nrl.navy.mil 

PI 

Mr.  Glenn  Harbaugh 

Nova  Research,  Inc. 

1900  Elkin  St,  Ste.  230 

Alexandria,  VA  22308 

804-761-5904  (V) 
glenn.harbaugh.ctr@nrl.navy.mil 

Site  Safety 
Officer 

Mr.  John  Breznick 

NAEVA  Geophysics,  Inc. 

P.O.  Box  7325 

Charlottesville,  VA  22906 

434-978-3187  (V) 
jbreznick@naevageophysics.com 

General 

Manager 
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APPENDIX  C.  DATA  FORMATS 


C.l  TEM  DATA  FILE  (*.TEM) 

These  data  files  are  a  binary  format  generated  by  a  custom  .NET  serialization  routine. 
They  are  converted  to  an  ASCII,  comma-delimited  format  in  batches  as  required.  Each 
file  contains  4  data  points,  corresponding  to  each  transmitter  (Tx)  cycle.  Each  data  point 
contains  the  Tx  transient  and  the  corresponding  12  receiver  (Rx)  transients  as  a  function 
of  time.  A  pair  of  header  lines  is  also  provided  for,  one  overall  file  header  and  one 
header  per  data  point  with  the  data  acquisition  parameters.  A  partial  example  is  provided 
below. 

Line  1  -  Eile  Header 


TargetID,  Bkgnd, Date, CPUms, GPSUTC, Lat, Lon, HAE, GPSFixQ, UTM_Zone, UTM_E, UTM 
_N, Elev, Heading, Pitch, Roll, Belt, BlockT, nRepeats, DtyCyc, nStk, AcqMode, Gat 
eWid, GateHOff , TxSeq, GateT, TxI_Z, RxlZ_TxZ, RxlY_TxZ, RxlX_TxZ, Rx2Z_TxZ, Rx2 
Y_TxZ, Rx2X_TxZ, Rx3Z_TxZ, Rx3Y_TxZ, Rx3X_TxZ, Rx4Z_TxZ, Rx4Y_TxZ, Rx4X_TxZ 

Line  2  -  Data  Point  Header 


4,0,5/28/2013,36985640.625,134934.30,38.409854552,-77.10894215,- 
33.570,4,18,315866.183,4253396.760,, 253.956,3.98442,-2.81274,2E- 
06,0.9,9,0.5,18,2,0.05,5E-05,1 


4 

0 

5/28/2013 

36985640 . 625 

134934.30 

38.409854552 

-77 . 10894215 

-33.570 

4 

18 

315866.183 

4253396.760 

253.956 
3.98442 
, -2 .81274 
2E-06 
0 . 9 
9 

0.5 

18 

2 

0.05 

5E-05 

1 


-  Target  ID 

-  Background  file  Boolean  (1  =  background) 

-  Collection  date 

-  Start  time  in  ms  on  CPU  clock 

-  UTC  time  of  data  collection 

-  GPS  Latitude  (decimal  deg.) 

-  GPS  Longitude  (decimal  deg.) 

-  Height-above-ellipsoid  (m) 

-  GPS  Fix  Quality 

-  UTM  Zone 

-  UTM  Easting  (m) 

-  UTM  Northing  (m) 

-  GPS  Elevation  (m)  -  intentionally  left  blank 

-  IMU  Yaw  (deg,  magnetic  North  referenced) 

-  IMU  Pitch 

-  IMU  Roll 

-  Time  step  for  transients  (seconds) 

-  Base  period  length  (seconds) 

-  Number  of  Tx  cycles  in  a  base  period 

-  Duty  cycle 

-  Number  of  base  periods  averaged  (or  stacked) 

-  Data  Acquisition  Mode  (Decimated  Decays) 

-  Gate  width  as  fraction  of  its  own  time 

-  Hold-off  time  (seconds)  for  first  data  point 

-  Tx  coil  ID  number 


C-I 


Line  3  -  First  Data  Line  in  First  Data  Point 


fffffffffffffffffffffffff 


2 .5E-05, 0.491852843863786,- 


0.0019447468039619,- 

0.00167008188106537,0.00125536061090642,0.000316940817640296,0.00032090 

2009700342,-0.000864236384734167,0.00112260623919521,- 

0.000175718325757418,0.000508878029254883,0.000640205921826291,0.001023 

92797989884,-0.000598618916918521 


C.2  LOCATION  AND  ORIENTATION  DATA  FILE  (*,GPS) 


, Latitude/Easting/Pitch, Longitude/Northing/Roll, HAE/Zone/Yaw, Samples/UT 
C , FQ_0 , FQ_1 , FQ_2 , FQ_3 , FQ_4 , FQ_5 

GPS,  38 .40  9854552, -77 .108  94215,-33.57  0,20,0,0,0,0,20,0 
UTM, 315866.183,4253396.760,18,134934.30 
IMU, 3. 98442, -2 .81274,253.956,20 

These  data  files  are  ASCII  format,  comma-delimited  files.  A  header  line  is  provided. 
Line  1  -  Header  information 


Line  2  -  Raw  GPS  data 


GPS 

- 

Data  Type  Identifier 

38.409854552 

- 

Latitude  (decimal  deg) 

-77 . 10894215 

- 

Longitude  (decimal  deg) 

-33.570 

- 

Height -above -ell ipso id 

(m) 

20 

- 

Number  of  samples  received 

0 

- 

Number  of  samples  with 

FQ  0 

0 

- 

Number  of  samples  with 

FQ  1 

0 

- 

Number  of  samples  with 

FQ  2 

0 

- 

Number  of  samples  with 

FQ  3 

20 

- 

Number  of  samples  with 

FQ  4 

0 

- 

Number  of  samples  with 

FQ  5 

Line  3  -  Computer 

Location  data 

UTM 

- 

Data  Type  Identifier 

315866.183 

- 

UTM  Easting  (m) 

4253396.760 

- 

UTM  Northing  (m) 

18 

- 

UTM  Zone 

134934.30 

- 

UTC  Time 

Line  4  -  IMU  Data 

IMU 

- 

Data  Type  Identifier 

3.98442 

- 

IMU  Pitch  (deg) 

-2 .81274 

- 

IMU  Roll  (deg) 

253.956 

- 

IMU  Yaw  (deg,  magnetic 

North 

20 

- 

Number  of  samples  received 

referenced) 
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C.3  FIELD  NOTES  FILE  (*.TXT) 


Cart  Rolled:  False 

Interference  with  Measurement:  False 
Recentered  Array:  False 
Data  Issues:  True 
Can't  Center  on  Target:  False 
4-inch  Aluminum  sphere 

These  data  files  are  ASCII  format,  comma-delimited  files.  No  header  line  is  provided. 
Lines  1  (-  6)  -  Standardized  button  information,  if  defined 
Final  Line  -  Comments,  if  any 


C-3 


C.4  ANOMALY  PARAMETER  OUTPUT  FILE 


The  UX-Analyze  Advanced  module  for  Geosoft’s  Oasis  montaj  will  be  used  to  analyze 
the  TEMTADS  data.  The  fitted  parameters  for  eaeh  investigated  anomaly  are  distributed 
as  an  Exeel  2010  spreadsheet,  but  an  exeerpt  is  given  in  .esv  format  below  for  referenee 
purposes.  A  header  line  is  provided  for  information  followed  by  a  109-line  bloek  for 
eaeh  anomaly.  The  first  line  of  eaeh  bloek  eontains  the  time  gate-independent  fit 
parameters  and  the  remaining  108  eontain  the  time  gate-dependent  parameters  for  eaeh 
anomaly. 

Anomaly_ID, Anomaly_X, Anomaly_Y, Anomaly  Amplitude, Fit_X, Fit_Y, Fit_Depth ( 
m) , Fit_Phi (deg) , Fit_Theta (deg) , Fit_Psi (deg) , Fit_Coherence, Time_Gate, Bet 
al , Beta2 , Beta3 

28,402751.00,4369521.75,234.34,402750.926,4369521.686,0.151,250.42,2.02 
,76.57,0.99612, , , , 

,,,,,,,,,,, 1, 1 . 47E+00, 1 . 05E+00, 1 . 08E+00 

,,,,,,,,,, ,108,2 . 46E-05, -1 . 69E-05, -1 . 60E-04 

33,402726.00,4369505.50,15.24,402725.835,4369505.588,0.422,96.25,16.45, 
5.26,0. 96448, , , , 

,,,,,,,,,, ,1, 1.71E+00, 1.23E+00, 1.18E+00 

,,,,,,,,,,,108,6 . 56E-04, -1 . 91E-03, -1 .57E-04 
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